In order to enhance the robustness of the 3D point cloud face recognition system for multiple expressions and multiple poses, a deep learning-based point cloud feature extraction network was proposed, namely ResPoint. The modules such as grouping, sampling and local feature extraction (ResConv) were used in the ResPoint network, and skip connection was used in ResConv module, so that the proposed network had good recognition results for sparse point cloud. Firstly, the nose tip point was located by the geometric feature points of the face, and the face area was cut with this point as the center. The obtained area had noisy points and holes, so Gaussian filtering and 3D cubic interpolation were performed to it. Secondly, the ResPoint network was used to extract features of the preprocessed point cloud data. Finally, the features were combined in the fully connected layer to realize the classification of 3D faces. In the experiments on CASIA 3D face database, the recognition accuracy of the ResPoint network is increased by 5.06% compared with that of the Relation-Shape Convolutional Neural Network (RS-CNN). Experimental results show that the ResPoint network increases the depth of the network while using different convolution kernels to extract features, so that the ResPoint network has better feature extraction capability.